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ABSTRACT 

Factors predicting vocabulary learning and reading comprehension of advanced language learners of 
English in a linear multimedia text were investigated in the current study. Predictor variables of interest 
were multimedia type, reading proficiency, learning styles, topic interest and background knowledge about 
the topic. The outcome variables of interest were vocabulary and reading comprehension scores. 
Participants were 69 undergraduates enrolled at the foreign language teaching department of a Turkish 
university. Participants were randomly assigned to three different forms of an authentic electronic text, 
which differed from each other based on the type of multimedia: (a) definition of words, (b) definitions 
coupled with pictures, and (c) definitions coupled with short movies. The participants were given the text to 
read for general comprehension and were given an unannounced vocabulary test along with a reading 
comprehension test. Multiple regression analyses with vocabulary scores and reading scores as the criterion 
variables and the independent variables as the predictors served to reveal whether a relationship existed 
between the independent and dependent variables. Findings suggest that annotation type, reading ability 
and prior topical knowledge are important variables contributing to vocabulary learning whilst reading 
ability and learning styles (visual score) are important variables contributing to reading comprehension in a 
hypermedia environment. 

Keywords: Multimedia/Flypermedia, media in education, computer assisted language learning, vocabulary 
learning, reading comprehension 

OZET 

Bu gahijma ileri diizey dil ogrencilerinin dogmsal hipermetin okuma ortamlannda sozciik ve okuma 
puanlarma etki eden faktorleri incelemektedir. Bagimsiz degi^kenler agiklayici not turu, okuma bilgisi, 
bili§sel ve algisal ogrenme stilleri, konuya ilgi ve on bilgi testlerinin sonuglandir. Bagimli degi^kenler, 
sozciik ve okuma testlerinin sonuglandir. Ara§tirmaya Tiirkiye’deki bir iiniversitede okuyan 69 adet 1. simf 
ingilizce ogretmenligi ogrencisi katilmi§tir. Katihmcilar, ozgiin bir elektronik metni okumak amaciyla 
yansiz atama yoluyla tig gmba aynlmi§lardir. Elektronik metinler birbirlerinden agiklayici notlann tiirii 
bakimindan farklidir. Ug farkh agiklayici not kullamlmi§tir: a) Sozciik tammlan, b) sozciik tammlan ve bu 
tammlarla ilgili resimler, c) sozciik tammlan ve bu tammlarla ilgili kisa filmier. Katilimcilardan anlama 
amaciyla metni okumalan istenmi^tir. Daha sonra onceden duyurulmami§ bir sozciik testi ile beraber 
okuma testi verilmi§tir. Degi^kenler arasindaki ili§kiyi gormek amaciyla yapilan goklu regresyon analizine 
sozciik ve okuma testlerinin sonuglan bagimli degi^ken olarak, yukarida sayilan diger degi^kenler ise 
bagimsiz degi^kenler olarak dahil edilmi^tir. Sonuglar, agiklayici not tiirii, okuma bilgisi ve konuyla ilgili 
on bilginin hipermetinde sozciik ogrenmeye katkisi olan, ote yandan okuma bilgisi ve gorsel ogrenme 
stillerinin okuma-anlamaya katkisi olan degi^kenler oldugunu gostermi^tir. 

Anahtar Kelimeler: (,'oklu ortamlar, egitimde medya, bilgisayar destekli dil egitimi, sozciik ogrenme, 
okuma-anlama 

INTRODUCTION 

Using multimedia or hypermedia as a supplemental medium for foreign/second language (L2) learning and 
teaching has received considerable interest. Notable numbers of multimedia software have been developed 
for teaching foreign languages since multimedia is considered a promising technology to facilitate L2 
learning and teaching (Chiquito, Meskill, & Renjilian-Burgy, 1997). Due to its integration of different 
media, multimedia could have positive effects on language learning (Kramsch & Andersen, 1999; Shea, 
1996). Along with the presentation of input in multiple forms, interaction provided by hypermedia and the 
use of authentic materials can make it an ideal language learning medium (Chiquito et ah, 1997; Heller, 
1990; Kramsch & Andersen, 1999). 
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Multimodality in hypermedia environments is basically provided with the aid of hypermedia annotations or 
glosses incorporated into a reading text in multiple forms of media. It is suggested that providing glossaries 
is better than simplifying a text in helping learners to cope with comprehension since simplification 
deteriorates the authenticity of the text (Luppescu & Day, 1993; Watanabe, 1997; Widdowson, 1984). 
Moreover, glosses increase flow of reading, independence from dictionaries and accuracy in finding 
meaning (Nation, 1990). Finally, they help readers confirm or disconfirm their vocabulary guesses from the 
context. 

Research studies on the effectiveness of hypermedia on L2 learning have mostly investigated the effects of 
multimedia glossing techniques on certain aspects of L2 learning such as vocabulary learning (Chun & 
Plass, 1996a; De Ridder, 2003; Hulstijn, Hollander, & Greidanus, 1996; Knight; 1994; Seghayer, 2001) and 
text comprehension (Ariew & Ergetin, 2004; Aust, Kelley, & Roby, 1993; Chun & Plass, 1995, 1996b; 
Davis & Lyman-Hager, 1997; Er?etin; 2003; Knight, 1994; Lomicka, 1998; Roby, 1999; Sakar & Ercetin, 
2005). 

With respect to vocabulary learning, previous studies suggest that glosses/annotations with word definitions 
and visuals (e.g., pictures and videos) help learners learn vocabulary more efficiently than annotations with 
word definitions alone (Chun & Plass, 1996a; Seghayer, 2001). However, empirical studies have revealed 
contradictory findings regarding the relative importance of the type of visual annotations. Chun and Plass 
(1996a) suggest that annotations which include word definitions coupled with pictures are more effective 
on vocabulary learning than annotations that include definitions coupled with videos. Seghayer’s (2001) 
investigation, on the other hand, has yielded the conclusion that definitions coupled with videos are more 
effective in facilitating vocabulary learning. 

When certain variables are added into the research design, notable changes in the results can occur. For 
instance, Knight (1994) investigated the effects of on-line dictionary access along with L2 verbal ability on 
vocabulary learning of learners of French. Results indicated that vocabulary learning outcomes varied 
according to the linguistic proficiency of the learners. 

As for reading comprehension, it is suggested that hypermedia environments have the potential to facilitate 
L2 reading comprehension since they provide additional information at both the word and the topical level 
(Lomicka, 1998; Martinez-Lage, 1997). In other words, electronic reading provides L2 readers with the 
opportunity to learn beyond the texts via textual and extratextual information contrary to the traditional 
reading. However, the relationship between hypermedia use and reading comprehension is somewhat 
indirect (Chun, 2001). Other variables, such as verbal ability and proficiency may interact with the effects 
of hypermedia (Ariew & Ercetin, 2004). These predictions stem from the reading studies conducted in LI 
and L2. These studies revealed that the ability to utilize better reading strategies to comprehend the text 
depends on the learners' foreign language reading proficiency (Devine, 1988), prior knowledge on the 
subject domain (Carrell, 1988), topic interest (Schiefele & Krapp, 1996) and learning styles (Bernhardt, 
1991). 

The variables that affect traditional reading and vocabulary learning are expected to be influential in 
hypermedia environments as well. Furthermore, providing several authentic multimedia symbol systems 
should create a more interesting learning environment serving learners with different learning styles. It is 
suggested that differences in learning styles should result in distinctive navigation patterns and differences 
in learning outcomes in hypermedia environments (Parkinson & Redmond, 2002). Literature provides 
studies with contradictory findings some establishing a relationship between individual learning styles and 
learning outcomes (Andris, 1996; Plass, Chun, Mayer, & Leutner, 1998) whilst others failing to establish a 
connection between learning styles and achievement (Calcaterra, Antonietti, & Underwood, 2004; Liu & 
Reed, 1994). Replication of the studies in different learning situations and with different learner 
populations may help to shed light on these issues. The current design aims to investigate whether selected 
explanatory variables predict foreign language vocabulary learning and reading comprehension in 
hypermedia environments at a practically significant level. 
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METHOD 

Participants 

The experiment was conducted with 69 (22 males and 47 females) freshman TEFL students studying at an 
English-medium university in Turkey. In order to encourage participation in the experiment, full 
participation was awarded with 5 % of students’ English Composition grades. Participants were randomly 
assigned to three groups each containing 23 students. 

Materials 

Treatment: Reading Software 

Using the reading software designed by Ariew (1999), a hypermedia text was generated. The software 
allowed integration of glosses to an electronic text collecting data with its built-in tracking tool, which 
recorded every single interaction of the readers with the material. That is, the frequencies of access to 
annotations, the order in which they are accessed and the amount of time spend on reading were recorded 
by the tracking tool. 

The original text of the software comes from an advanced reading book by Smith and Mare (1997). The 
reading passage was chosen based on a topic interest questionnaire given to participants before. A pen-and- 
paper version of the original text was piloted with 20 senior students who did not participate in the original 
study in order to determine the words to be annotated. All unknown words were annotated. For each word, 
4 pictures and 3 videos were found. Four English teachers selected the most suitable videos and pictures for 
each word. Each picture was standardized to a size of 450 pixels width and 303 pixels height. The videos 
used were similar to each other in length and quality. The mean length of the videos was 13.7 seconds (SD: 
1.07). The American Fleritage Dictionary (2000) was used as the source of all the word definitions. 

Three parallel forms of the electronic text were prepared. The first form included only textual definitions, 
which provided the definition of a highlighted word or phrase, and its grammatical form -noun, adjective, 
verb, etc appears. For example, the textual annotation for the word “doomed” appears as ‘adj. - Condemned 
to ruination or death’. The second version of the text included an associated image of the word in addition 
to the definition provided in the first group. The third version of the text included an associated movie and 
the definition provided in other groups. In picture and video groups, participants were free to choose either 
one type of annotation or both types provided. Annotations were presented successively when they were 
clicked rather than simultaneously. 

The text was presented in a linear order in 9 pages and students were allowed to go forward and backward 
in the text by clicking on one of two arrow-buttons provided at the bottom of the screen. The design had a 
navigation map at the top-left side of the screen that indicated which page the student was reading and how 
many pages there were left to read. 

Vocabulary Test 

Overall vocabulary scores were computed based on 3 subtests. They were prepared by the researcher based 
on the tests used by Knight (1994), Rott (1999) and Waring and Takaki (2003). All subtests were piloted 
for item development. Items with insufficient item facility, item discrimination and distractor efficiency 
indices were modified. The first subtest included a checklist in which students were asked to indicate the 
words they remembered from the text. The checklist was comprised of 42 target items and 30 fillers. The 
second subtest involved students writing the LI equivalents or synonyms of every target word. Since this 
subtest would allow subjectivity in scoring, two independent raters scored the papers and an interrater 
reliability of .95 was found for the pre-test, .98 for the immediate post-test and .98 for the delayed post-test. 
The third subtest was a multiple-choice test whose first half involved synonym recognition and the second 
half involved definition recognition. The reliability coefficients obtained from the actual administration 
revealed that alpha of the pre-test was .76, that of immediate post test was .72 and that of the delayed post 
test was .69. 

Reading Comprehension Test 

The reading comprehension test consisted of a multiple-choice and a true-false section. The true-false part 
consisted of 22 questions and each question had a third choice called “no information”. This reduced the 
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chance of getting the right answer by guessing alone from 50 % to 33 %. The multiple-choice part 
consisted of 12 questions, 4 distractors for each. In both parts, grammatically parallel and correct 
distractors were provided. Moreover, incidental insignificant information and details were not tested. The 
test was developed by the researcher with the assistance of an expert, and edited by an American colleague 
to eliminate odd structures or unidiomatic language. Administration procedures including speed and time 
control were determined after a pilot study. The actual data revealed that the Cronbach’s Alpha of the 
multiple choice test was .79 and that of true-false part was .70. 

Prior Knowledge Test 

The prior knowledge test consisted of two parts. In the first part, participants were asked to write down 
every single thing they knew about the topic. A total of 31 propositions were mentioned by the participants 
in the essays and each coherent and acceptable proposition was given 1 point. An independent rater was 
trained and an interrater reliability coefficient of .97 was found between the scorers on the first part. The 
second part consisted of 5 open-ended questions and 2 precise completion items. The highest possible score 
was determined to be 16 for the second part. A detailed scoring rubric was prepared and an independent 
rater was trained. An interrater reliability coefficient of .96 was found between the researcher and the 
independent scorer on the second part. The cumulative scores of the two parts were used as the prior 
knowledge score. 

Nelson - Denny Reading Test 

In order to see whether the groups were equal with regard to their reading ability, the Nelson-Denny 
Reading Test prepared by Brown, Fisho and Hanna (1993) was used. It is a standardized reading survey for 
high school and college students and adults, which measures vocabulary development, comprehension and 
reading rate. The test was administered to all the participants at the same time and in the same room 
according to the instructions given in the Nelson-Denny test manual. 

Learning Style Questionnaire 

Rebecca Oxford’s (1993) ‘Style Analysis Survey (SAS)’ was used as the learning style assessment 
instillment in the study. The material provides individuals’ overall style preferences focusing on their 
general approach to learning and working. SAS has a Cronbach Alpha of .87 in testing with 468 language 
students at the university level in a U.S. university. 

Topic Interest Questionnaire 

On the very same sheet with the prior knowledge test, a topic interest test was also given to the students. 
The test was adapted from Schiefele and Krapp (1996). The questionnaire consisted of value-related and 
feeling related valences. To calculate value-related valences, students were asked to rate the topic on its 
usefulness, meaningfulness, importance and worthiness on a 4-point scale, 4 showing complete agreement 
and 1 showing complete disagreement. To calculate feeling-related valences, students were asked how they 
would feel, when they read the text by rating again on a 4-point scale, 4 implying complete agreement with 
a specific feeling and 1 implying complete disagreement with it; i.e., boredom, interest, indifference, 
involvement and engagement. For each subject, a score of topic interest was computed by adding feeling- 
related and value related scales (Schiefele & Krapp, 1996; 146). The highest possible score was 36. 

Procedure 

Participants were given the Nelson-Denny Reading test, the prior knowledge test and vocabulary pre-test 
two weeks before the treatment. They were given the learning style questionnaires a week before the 
treatment. The treatment was given at one of the computer laboratories of the university, which was 
designed for simultaneous processing of 25 computers each connected to a local area network (LAN). 
Participants were given the treatment successively on the very same day. Before the reading session, the 
researcher oriented students with the reading software by providing a demonstration on how the software 
worked through a data projector. Each group was given exactly the same instructions and was told that they 
should read the text carefully since they would have a reading comprehension test after the treatment 
session. After the treatment, participants were given the unannounced vocabulary test, the announced 
reading comprehension test and a background questionnaire successively. Within four days following the 
treatment 63 of the 69 students were given semi-structured interviews. Exactly 3 weeks after the treatment, 
again unannounced, all students were given the vocabulary test again. 
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Data Analysis 

Since randomization was realized, the groups were considered to be theoretically equal (Ary, Jacobs, & 
Razavieh, 1996). However, a further inspection of the groups was also carried out. The groups were 
compared using one-way analyses of variance (ANOVA) on vocabulary pre-test scores, general point 
average (GPA), age, prior topical knowledge, topic interest, L2 reading ability, PC aptitude, WWW use, 
learning style, and frequency of lookup behavior to determine whether the groups differed from each other 
in terms of those variables at a probability value of .05 or below. The assumptions of ANOVA were 
checked before running the analyses. The groups were not different from each other at the inception of the 
study at a probability value of .05. 

In order to control for the vocabulary pre-test scores, gained vocabulary scores were calculated. All 
variables were put into a bivariate correlation calculation in SPSS 14.0 for windows. Explanatory variables 
that had moderate or high correlations with response variables were selected, and multiple regression 
analyses with each response variable were conducted. Assumptions of normally distributed errors, linearity, 
multicollinearity and uncorrelated residual term were checked before interpreting the analyses. 

RESULTS 

Vocabulary Learning 
Immediate Post-test Scores 

Predictor variables that had high correlation with gained vocabulary post-test scores were language 
proficiency (r=,323, p<.007) and prior topical knowledge (r=.355, p<.003). Text format was used as a 
dummy variable in the analysis. Variables were entered into the analysis successively. Language 
proficiency scores explained 10 % of the vocabulary post-test scores (R 2 =. 104, F=7.801, p<.007). The next 
variable, annotation type, created an R squared change of .16. More specifically, it explained an additional 
16 % in the criterion variable (R 2 change=.16, F change=7.076, p<.002). The final variable, prior 
knowledge score, explained an additional 4.6 % in the criterion variable (R 2 change=.046, F change=4.309, 
p<.042). The regression model with these three variables accounted for 31 % of variance in the criterion 
variable. In other words, above variables explained 31 % of immediate vocabulary post-test scores. The 
regression equation is as follows: Immediate post-test score= 0.258 (language proficiency) + 0.241 
(annotation type) + 0.228 (prior topical knowledge). 

Further analysis revealed that the groups performed differently based on the annotation type (F 2j 66=9.819; 
p<.001). Multiple comparisons with Scheffe test revealed that both the picture and video group performed 
significantly better than the definition only group in terms of immediate post-test scores. However, there 
was no difference between the picture and the video group. 

Delayed Post-test Scores 

The only predictor variable that had high correlation with delayed vocabulary post-test scores was language 
proficiency (r=,319, p<.008). Even though prior topical knowledge had a significant correlation with the 
criterion variable (r=,278 p<.021), it did not create a significant R 2 value in the hierarchical regression 
analysis. Text format was used as a dummy variable in the analysis again. The analysis revealed that 
language proficiency scores explained 10 % of variability in the criterion variable (R 2 =.102, F=7.590, 
p<.008). Annotation type, created an R squared change of .12. In other words, it explained an additional 12 
% in the criterion variable (R 2 change=.116, F change=4.828, p<.011). Overall, the model with these two 
variables accounted for 22 % of variance in the criterion variable. More specifically, the variables 
explained 22 % of delayed vocabulary post-test. The following regression equation evolves: Delayed post¬ 
test score= 0.306 (language proficiency) + 0.163 (annotation type). 

Further analysis was conducted with delayed post-test scores as well. Similar to above results, groups 
performed differently based on the annotation type (F 2> 66 =4.822; p<.011). Multiple comparisons with 
Scheffe test revealed that the video group performed significantly better than the definition only group. 
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Reading Comprehension 

Potential explanatory variables that had high correlation coefficients with the criterion variable were 
reading ability (r=.386, p<.001) and learning styles (SAS visual score) (r=.248, p<.04). Annotation type 
was used as a dummy variable. A hierarchical regression analysis with the reading comprehension scores as 
the criterion and above variables as the predictors was conducted. Results revealed that language 
proficiency scores explained 15 % of variability in the criterion variable (R 2 =. 149, F=11.760, p<.001). The 
second variable, SAS visual score, explained an additional 6 % variability in the criterion (R 2 change=.059, 
F change=4.887, p<.03). Annotation type did not have a significant R 2 value in the analysis. In sum, 
language proficiency and SAS visual score explained 21 % of variability in reading comprehension scores. 
The regression equation evolving from this analysis is as follows: Reading comprehension score= 0.383 
(language proficiency) + 0.242 (SAS visual score). 

DISCUSSION 

Findings of the study suggest that when advanced language learners are under scrutiny, predictors of 
vocabulary learning in hypermedia reading environments are annotation type, reading ability and prior 
topical knowledge. Moreover, findings suggest that predictors of reading comprehension are reading 
ability and SAS visual score. 

These findings confirm the results of previous studies. More specifically annotation type is an important 
variable contributing to vocabulary learning (Chun & Plass, 1996a; Seghayer, 2001) along with linguistic 
proficiency (Knight, 1994). Moreover, as suggested by Devine (1988), the ability to comprehend the text 
primarily depends on the learners' foreign language reading proficiency. Finally, in terms of learning 
styles, the findings follow a path similar to those of Andris (1996) and Plass et al (1998) maintaining that 
there is a relationship between individual learning styles and learning outcomes. Within the scope of the 
current study, one could claim that hypermedia environments may provide learning opportunities for 
learners with different visual learning styles. 

Combining text with visuals regardless of the type of visual used is more effective in facilitating 
vocabulary learning than providing only definitions of words. In this respect, findings are in line with the 
Generative Theory of Multimedia Learning (Mayer, 1997; 2001), particularly with the dual channels 
assumption, which suggests that it is better to present an explanation in words and visuals than solely in 
words. 

Treatment of the study had low element interactivity, that is, elements in the material could be learnt 
successively rather than simultaneously since they did not interact. So, the intrinsic cognitive load was 
low. Sweller and his colleagues suggest that extraneous cognitive load is a problem in cases of high 
element interactivity. They claim that manipulating the instructional design might have no beneficial 
outcomes when the element interactivity is low (Sweller, 1994; Sweller, van Merrienboer, & Paas, 1998). 
The current study suggested that students exposed to visual annotations were better in vocabulary tasks 
than those who were merely exposed to verbal annotations even though the element interactivity of the 
material is low. Thus, extraneous cognitive load could be a problem also for low element interactivity 
materials for advanced language learners. 

Several pedagogical implications can be suggested based on the current design and relevant literature. In 
hypermedia reading environments, the interaction between the reader and the text provides individualized 
learning and promotes learner autonomy. Learners can have control over their learning process and learn 
at their own pace. Therefore, both slow learners and fast learners can make use of hypermedia 
environments efficiently because of the interaction they provide. Second, the presentation of authentic 
input is made easier with hypermedia software. Besides, learners are not only presented with real 
language, but also with original means to deal with the authentic language. Thus, multimedia are more 
feasible in creating natural L2 learning tasks than text simplification methods that may result in 
inauthenticity. Third, in order to make better use of hypermedia environments, L2 readers need to add new 
strategies to their repertoire to be independent during the reading process. In this respect, familiarizing L2 
learners with hypermedia experience and training them can be quite feasible. Finally, professionals 
involved in material development should consider the importance of visual aids in language teaching 
through hypermedia. 
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The study poses several limitations; thus, the findings should be considered with caution. First, the target 
population of the study was EFL students who learn English for academic purposes. This study should be 
replicated in other learning contexts with students from different levels to generalize findings to a larger 
target population. Second, further studies with larger sample sizes must be conducted to investigate 
whether a really significant variation existed in the population. Third, the use of a pre-test affected the 
generalizability of this study: the results cannot be generalized to people who are not pre-tested. Finally, 
participants were exposed to a non-traditional treatment, but tested with traditional testing methods in the 
current study. Employing on-screen tests where visual elements are incorporated would be more suitable 
for use in hypermedia environments. 
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